WORLD INTELLECTUAL PROPERTY ORGANIZATION 
International Bureau 




INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(51) International Patent Classification : 
G02B 21/00 


A2 


(11) International Publication Number: WO 00/43819 
(43) International Publication Date: 27 July 2000 (27.07.00) 


(21) International Application Number: PCT/GBOQ/OOt 42 

(22) International Ffling Date: 17 Januaiy 2000 (17.01.00) 

(30) Priority Data: 

9901365.8 22 January 1999 (22.01.99) GB 

(71) Applicant (for all designated States except US)i ISIS INNO- 

VATION LIMITED [GB/GB]; 2 South Paries Road. Oxford. 
Oxfordshire OXl 3UB (GB). 

(72) Inventors; and ^--s^ — ^ ^ 
(75) Inventors/Applicants (for US only)i^lLSON, Tony [GB/OBJ? 

18 Jeune Street, Oxford. OxfortShire 0X4 IBN (GB). 
NEIL, Mark, Andrew, Aquilla [GB/GB]; 22 Poplar Road. 
Botley. Oxford, Oxfordshire 0X2 9LB (GB). JUSKAmS, 
Rimvydas [LT/GB]; 3 Mairiott Close, Oxford, Oxfordshire 
0X2 8NT (GB). 

(74) Agents: PERKINS, Sarah et al.; Stevens, Hewlett & Perkins, 
Halton House, 20/23 Holbom. London ECIN 2JD (GB). 


(81) Designated States: CA, JP. US, European patent (AT, BE. CH, 
CY. DE, DK, ES, PI. FR, GB, GR, IE. IT. LU. MC. NL. 
PT. SE). 

Published 

Without international search report and to be republished 
upon receipt of that report. 



(54) Title: CONFOCAL MICROSCOPY APPARATUS AND METHOD 




(57) Abstract 

The confocal microscope has two matched light sources: a first light source (1) and a second light source (8). The light sources (1,8) 
are arranged to illuminate opposite sides of a modulating mask (6)^ The light either reflecting from or passing through the modulating mask 
(6) is then used to illuminate an object O supported on a mount (5). The microscope is arranged so that the object O is mounted on the 
opposite side of the mask (6) to a camera (7) such that light reflected from the object O passes through the modulating mask (6) before 
being captured by the camera (7). Subtraction of the image produced using the second light source (8) from the image produced using the 
first light source (1) generates a confocal image that contains substantially less noise than is possible with available confocal microscopy 
apparatus. 
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CONFOCAL MICROSCOPY APPARATUS AND METHOD 

The present invention relates to a confocal microscopy apparatus 
and method and in particular to confocal microscopy apparatus that 
5 enables real-time imaging to be perfonned. 

Confocal microscope systems were originally designed to generate 
a substantially pure confocal image by scanning a diffraction limited spot of 
light across the object being imaged. Light reflected from separate 
scanned points of the object was sequentially received by a series of 
1 0 photodetectors to gradually build up a 2-D confocal image of the object. 
Such confocal systems have the disadvantage that a very bright light 
source such as a laser is necessary and real-time imaging could not easily 
be performed because of the time required to scan over the surface of the 
object. 

15 More recent developments have focused on ways of increasing the 

speed with which confocal images can be generated to enable real-time 
imaging, in W097/31282 a confocal microscope is described in which a 
mask is used to generate a combined confocal and non-confocai image. 
The non-confocal image is subsequently subtracted from the combined 

20 image to provide a substantially pure confocal image. The mask in 
W097/31282 consists of different, separated regions with a first region 
being substantially transparent to incident light and unpattemed and a 
second region being patterned, for example with an irregular array of spots 
that are opaque to incident light Light that passes through the first region 

25 illuminates the object to generate a conventional image of the object. On 
the other hand, light that passes through the second region produces a 
pattern of the mask on the surface of the object that in turn generates a 
combined confocal and conventional image of the object. The mask is 
spun so that the first and second regions of the mask alternately transmit 

30 the illurnination to the object and so generate alternate images of the 

object. The altemate images are then subtracted from one another so that 
the conventional image is removed from the combined image to leave only 
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the confocal image. 

An alternative tlieoretical design of a confocal microscope is 
described in a paper that appeared in the Journal of Microscopy, Vol. 189, 
Pt 3, March 1998 entitled "Theory of confocal fluorescence imaging in the 
5 programmable array microscope (PAM)", Verveer et al. The programmable 
array microscope described in this paper consists of a spatial light 
modulator (SLM) in the form of a digital micromirror device (DMD). The 
individual pixels of the SLM are programmed to generate an arbitrary 
pattern of conjugate illumination and detection apertures and two separate 

1 0 cameras are used, one positioned to view active ('on') micromin-ors of the 
SLM and to receive the confocal reflected light from the object and one to 
view inactive (*ofP) micromirrors and to receive a nonconjugate image of 
the object. However, significant difficulties have been encountered in 
developing the theoretical design to a working model and the design has 

1 5 never been made to work convincingly. 

The present invention seeks to provide improved confocal 
microscopy apparatus that enables real-time imaging of an object and 
generates images containing less noise than is generally possible with 
current systems. 

20 The present invention provides confocal microscopy apparatus 

comprising means for directing light to a specimen; at least one mask for 
encoding light incident on the specimen and for decoding light from the 
specimen to thereby generate a first image of the specimen consisting of a 
confocal image superimposed on a non-confocal image; and detection 

25 means for detecting said first image characterised in that there is further 
provided encoding means for encoding light to or from the specimen in 
reverse with respect to the encoding of the at least one mask to thereby 
generate a second image of the specimen consisting of a non-confocal 
image from which a confocal image has been removed. 

30 Ideally, analysing means are provided for generating a confocal 

image of the specimen using the first and second images. 

In a preferred embodiment a single mask is provided that encodes 
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light incident on the specimen and decodes light from the specimen. The 
encoding means may consist of reflective patteming on one surface of the 
mask. 

The microscopy apparatus may Include one or more light sources. 
In particular, two light sources may be provided to illuminate each side of 
the mask respectively. Furthermore, the detection means may detect both 
the first and second images. 

Alternatively, a second detection means may be provided for 
detecting the second image, the first and second detection means being 
located on opposite sides of the mask. The second detection means may 
be located to receive light reflected fi^m said reflective patterning. 

The mask may be patterned with equidistant light transmissive 
stripes and light barring stripes or with a random patteming. Also, the 
mask may be mounted for rotation about the optical axis of the apparatus. 

In an alternative aspect the present invention provides a confocal 
microscopy method comprising providing illumination from a first light 
source; encoding light incident on a specimen and decoding light from the 
specimen by means of one or more masks; generating a first image of the 
specimen consisting of a confocal image superimposed on a non-confocal 
image; and detecting said first image characterised by encoding light to or 
from the specimen in reverse with respect to the encoding of the one or 
more mask and generating a second image of the specimen consisting of a 
non-confocal image from which a confocal image has been removed. 

Ideally, the method further comprises generating a confocal image 
from said first and second images. Preferably, the second image is 
subtracted fl-om the first image to extract the confocal image. 

In a preferred embodiment a single mask is used to encode light to 
the specimen and to decode light from the specimen. Also, the second 
image may be generated by reflecting light off reflective patterning on one 
surface of said single mask. The mask may be rotatable. 

Embodiments of the present invention will now be described by way 
of example with reference to the accompanying drawings, in which: 
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Figure 1 is a schematic diagram of a first embodiment of confocal 
microscopy apparatus in accordance with the present invention; 

Figure 2 Is a schematic diagram of a second embodiment of 
confocal microscopy apparatus in accordance with the present invention; 
5 Figure 3 is a schematic diagram of a third embodiment of confocal 

microscopy apparatus in accordance with the present invention; 

Figure 4 Is a schematic diagram of a fourth embodiment of confocal 
microscopy apparatus in accordance with the present invention; 

Figures 5a and 5b are combined confocal and conventional images 
10 of a transistor at a first focal plane produced using the apparatus of Figure 
1; 

Figures 5c and 5d are respectively a confocal image and a 
convention image extracted from the images of Figures 5a and 5b; 

Figures 6a and 6b are combined confocal and conventional images 
15 of a transistor at a second focal plane produced using the apparatus of 
Figure 1; and 

Figures 6c and 6d are respectively a confocal image and a 
convention image extracted from the images of Figures 6a and 6b, 
Microscopy apparatus suitable for use in generating real-time 

20 confocal images is shown in Figure 1 . The microscopy apparatus includes 
a first light source 1 with an associated collimating lens 2 and a beam 
splitter 3 for directing light from the first light source 1 through an objective 
lens 4 to an object O or specimen supported on a mount 5. The beam 
splitter 3 may be a semi-silvered miaor, as shown in Figure 1. 

25 Alternatively, a polarising beam splitter may be used in combination with a 
quarter waveplate located between the mask 6 and the object O so as to 
minimise the effect of spurious reflections. In the case of fluorescence 
microscopy, a dichroic beam splitter may conveniently be used. 

Between the beam splitter 3 and the objective lens 4 a mask 6 is 

30 provided across the main optical axis X of the microscopy apparatus such 
that a first surface of the mask 6a is illuminated by the first light source 1 . 
The mask 6 is encoded with a predetemiined pattern that modulates 
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spatially in the plane of the mask the light from the first light source 1. The 
modulation may be intensity, phase or polarisation modulation. In the 
following description reference will be made to intensity modulation, 
however, it will be understood that polarisation may be substituted for 
5 intensity or phase throughout. The mask 6 has transparent and opaque 
patterning that extends through the mask from Its first surface 6a to Its 
opposing second surface 6b. The patterning covers substantially all of the 
first and second surfaces of the mask. For example, as indicated in Figure 
1 , the patterning may be in the form of regular stripes of substantially 

1 0 opaque material separated by stripes of substantially transparent material. 
Alternatively the patterning may be formed by an anray of small spots or a 
wholly irregular pattern. In order to ensure substantially zero cross- 
correlation between individual points or pixels of the mask, the patterning 
may be determined using finite length binary time sequences such as time- 

1 5 shifted complementary Golay sequences with the number of sequences in 
the set limited to the sequence length. In the case of the mask shown in 
Figure 1, the ratio of the transparent and opaque patterning is preferably 
1 :1 so as to maximise the light budget of the microscopy apparatus. Where 
the ratio is other than 1:1, the difference in the light transmitted/blocked by 

20 the patterning should be accommodated by normalisation factors employed 
in the extraction of the confocal image. The size of the patterning of the 
mask 6 is determined by the optical arrangement of the microscopy 
apparatus to the extent that the patterning must be resolvable on the object 
O. 

25 The mask 6 is mounted with its normal at a small angle, for example 

a few degrees, to the main optical axis X of the apparatus. The angle is 
sufficiently small that it has only a nominal effect to the final imaging of the 
mask pattern on the object. Where the patterning of the mask is fixed, 
preferably the mask is mounted on an axle (not shown) for rotation about 

30 its normal. However, the mask 6 may be fixed so that no rotation is 

pemnitted depending upon the type of patterning employed for the mask 6 
and in particular where the patterning can be cyclically altered such where 
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a spatial light modulator (SLM) is used as the mask. 

The beam splitter 3 Is positioned to reflect light from the light source 
1 towards the mask 6 and to allow light from the mask 6 to pass through to 
a camera 7. The camera 7 Is preferably a CCD camera, however, 
5 alternative two dimensional image detectors may be employed such as a 
simple array of photodetectors. 

A second light source 8 and an associated collimating lens 9 are 
also provided off the main optical axis of the apparatus so as to illuminate 
the second surface 6b of the mask that faces towards the object O. The 

1 0 second surface 6b of the mask bears reflective patterning that is 

substantially identical to the opaque patterning of the first surface of the 
mask. Preferably, where the opaque patterning of the mask extends 
through the thickness of the mask, the opaque patterning Is made reflective 
at least on the second surface 6b of the mask. In this way it is ensured that 

1 5 the reflective patterning is identical to the opaque patterning which encodes 
light from the first light source 1 . Hence, the second surface 6b of the 
mask 6 presents a reflective patterning to light from the second light source 
8. 

With the apparatus shown in Figure 1 , ideally the first and second 
20 light sources 1 , 8 illuminate the mask 6 with substantially identical intensity. 
Where necessary, though, predetermined normalisation factors can be 
employed during extraction of the confocal image to accommodate any 
differences between the intensities of the two light sources. Although 
Figure 1 shows two separate light sources alternatively, a single light 
25 source may be used that can be re-directed to illuminate either the first or 
second faces of the mask 6. This has the advantage of ensuring matched 
illumination of the mask. 

The light sources are preferably in the form of light emitting diodes 
(LEDs). However, alternative light sources sudh as incandescent lamps, 
30 arc lamps or lasers may be employed. Arc lamps are particularly desirable 
where fluorescent imaging of biological samples is to be perfomied. 

In use, the object O is alternately illuminated by the first and second 
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light sources 1 and 8. In the case of light from the first light source 1 , the 
light is reflected by the beam splitter 3 and is encoded by the mask 6 as it 
passes through the mask. The light thus fomis an image of the opaque 
patterning of the mask on the object O at a predetermined plane. The 
5 encoded light is then reflected by the object O back through the mask 6 to 
the beam splitter 3. As the reflected light returns through the mask 6, the 
light is decoded by the patteming of the mask. The decoded light then 
passes through the beam splitter 3 to the camera 7. 

The Image, referred to as the first or positive image, received by the 

1 0 camera 7 is thus a combination of a conventional image Iconv superimposed 
with a confocal image Iconf with the confocal image being produced from the 
encoded light that is focused on the object at the focal plane and is 
accurately decoded by the mask 6. On the other hand, light from the 
second light source 8 is reflected by the reflective patterning of the mask 6 

15 at the second surface 6b to the object O to produce a reverse image of the 
opaque patterning on the object O at the focal plane. Thus, the encoding 
of the light from the second light source 8 is the reverse of the encoding of 
the light from the first light source 1 . The light encoded by the reflective 
patterning is reflected by the object O back to the second surface 6b of the 

20 mask 6 and thence to the camera 7. The image, referred to as the second 
or negative image, received by the camera 7 is thus a conventional Image 
Iconv minus the confocal image Iconf as the confocal image has not been 
transmitted by the mask to the camera7. 

The positive and negative images produced using the first and 

25 second light sources are then transferred to an analyser 10 which subtracts 
the negative image from the positive image, i.e. (Iconv+IconfHIconv-Iconf). The 
resultant image 2 W is a confocal image having twice the intensity of 
confocal images produced using previous confocal microscopy systems. 
This in tum significantly increases the signal to noise ratio in the confocal 

30 images produced using the microscopy apparatus and method described 
above. The conventional image Iconv may also be extracted by the analyser 
10 by the addition of the positive and negative images. The conventional 
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image is particularly useful in navigating around the object being imaged. 
As mentioned above, the positive and negative images may be normalised 
where necessary to accommodate differences In the intensity of the 
illuminating beams. 
5 It will, of course, be apparent that where real-time imaging is 

required and alternate positive and negative images of the object are 
generated, each positive image and each negative image can be used 
twice with its corresponding neighbours in time. Thus, the microscopy 
apparatus is particularly suited to the confocal imaging of dynamic systems. 

1 0 Figure 5a is a positive image. (Iconv+Iconf). of a semiconductor 

transistor using the microscopy apparatus and method shown in Figure 1 
using a polarising beam splitter and LEDs as light sources. Figure 5b is the 
negative image, (Iconv-IconO, of the same transistor at the same focal plane. 
In Figure 5c the confocal Image extracted from the positive and negative 

15 images of Figures 5a and 5b is shown and Figure 5d shows the extracted 
conventional image. It will be immediately apparent from Figure 5c that the 
microscopy apparatus describe above is able to produce a particularly clear 
and strong confocai image. Figures 6a to 6d are further positive, negative, 
confocal and conventional images respectively of the same transistor at a 

20 second focal plane. 

The images shown in Figures 5 and 6 were produced using a 
rotating disc having alternate transparent and opaque stripes, each stripe 
being approximately SOjim wide. The mask was positioned with its normal 
at an angle of approximately 10° to the optical axis of the apparatus. In 

25 order to produce the confocal images at video rate, for example 25 Hz, 
each light source was operated at a frequency of 25/2 Hz. Of course the 
object could be illuminated at faster speeds, if necessary or at much slower 
speeds where real-time imaging is not required. Depending upon the type 
of light source used, the light source itself may be turned on and off at the . 

30 desired speed or alternatively, the output of the light source may be 
shuttered to control the illumination of the object. Rotation of the disc at 
high speed enables the pattern of stripes on the disc to be removed from 
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the final positive and negative images as tlie images can be integrated to 
average out the pattern of stripes. 

In an alternative microscopy apparatus as shown in Figure 2, a 
single light source 1 is used instead of two light sources. As in the previous 
5 embodiment the second surface 6b of the mask is reflective and the normal 
of the mask is set at a small angle, for example between 2° and 15*^. to the 
main optical axis X of the apparatus. However, in this case the second 
light source is replaced by a second camera 1 1 . As before a positive 
image is recorded by the first camera 7 however the negative image is 

10 simultaneously recorded by the second camera 1 1 from light reflected from 
the second surface of the mask 6. Both images are transmitted to the 
analyser 1 0 which then extracts the confocai image from the positive and 
negative images. This arrangement has the advantage that both images 
are produced simultaneously and so the confocai images can be produced 

15 approximately twice as fast. Also, the light budget is maximised as all 

available light is recorded either by the first or second cameras. However, 
this arrangement introduces greater difficulties with alignment as compared 
with the two light source system. 

In Figure 3 a further altemative embodiment is shown in which two 

20 light sources are used. This embodiment is similar to the first embodiment 
except that the camera 7 is moved so that it only receives images reflected 
from the second surface 6b of the mask. With this arrangement the first 
light source 1 generates the negative image and it is the second light 
source 8. on the same side of the mask as the camera 7, that generates 

25 the positive images. A fourth embodiment is shown in Figure 4 which is 
similar to Figure 2 in that the apparatus includes two cameras 7 and 11. 
With this altemative arrangement the first camera 7 receives the negative 
image and the second camera 1 1 , on the same side of the mask 6 as the 
light source 8 receives the positive image. 

30 Although reference herein has been to the generation of a two- 

dimensional confocai image of an object, it will be appreciated that three- 
dimensional confocai images may be produced using the microscopy 
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apparatus and method described above by creating a plurality of two- 
dimensional confocal images at different focal planes tiiat are used to 
construct the three-dimensional image. 

It will be apparent that the microscopy apparatus and method 
5 described above is suitable for most situations where confocal images are 
required. However, the apparatus and method is particularly suited for 
real-time imaging of biological processes and quality inspections for 
example of semiconductor chips. Although reference has been made 
throughout to light being reflected by the object, in many cases the object is 

10 not highly reflective in which case light scattered from the object is used to 
extract the confocal image. Also in the imaging of biological processes the 
incident light is not reflected by the object but rather is used to stimulate an 
equivalent pattern of fluorescence. The incident light is encoded in the 
manner described above and this pattern of light incident on the biological 

15 material stimulates fluorescence. The pattern of fluorescent light produced 
by the biological material is then decoded and positive and negative 
images are produced that can be subtracted to extract the confocal image 
in exactly the same way as described above. 

Also, the confocal microscopy apparatus may be implemented in a 

20 conventional microscope through the addition of the encoded mask and 
either a second light source or a second camera. 

It will of course be understood that altemative arrangements of 
optical elements, in particular altemative or additional lens arrangements, 
may be employed without departing from the spirit of the present invention. 

25 Furthermore, although reference has been made herein to a single mask 
having a patterned reflective surface it will be apparent that two or more 
transmission masks may be employed with one of the masks having a 
patteming that is the reverse of the patterning of one of the other masks. 
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CLAIMS 

1 . Confocal microscopy apparatus comprising means for directing light 
to a specimen; at least one mask for encoding light incident on the 

5 specimen and for decoding light from the specimen to thereby generate a 
first image of the specimen consisting of a confocal image superimposed 
on a non-confocal Image; and detection means for detecting said first 
image characterised In that there is further provided encoding means for 
encoding light to or from the specimen in reverse with respect to the 
10 encoding of the at least one mask to thereby generate a second image of 
lli^|the specimen consisting of a non-confbcal Image from which a confocal 
Image has been removed. 

2. Confocal microscopy apparatus as claimed in claim 1 , wherein there 
15 is further provided analysing means for generating a confocal image of the 

specimen using the first and second images. 

3. Confocal microscopy apparatus as claimed in either of clams 1 or 2, 
wherein a single mask is provided that encodes light incident on the 

20 specimen and decodes light from the specimen. 

4. Confocal microscopy apparatus as claimed in claim 3. wherein the 
encoding means consists of reflective patterning on one surface of the 
mask. 

25 

5. Confocal microscopy apparatus as claimed in claim 4, further 
including one or more light sources. 

6. Confocal microscopy apparatus as claimed in claim 5, wherein two 
30 light sources are provided illumination to each side of the mask 

respectively. 
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7. Confocal microscopy apparatus as claimed in claim 5. wherein the 
detection means detects the first and second Images. 

8. Confocal microscopy apparatus as claimed in 4, further providing a 
5 second detection means for detecting the second image, the first and 

second detection means being located on opposite sides of the mask. 

9. Confocal microscopy apparatus as claimed In claim 8, wherein the 
second detection means is located to receive light reflected from said 

10 reflective pattemlng. 

1 0. Confocal microscopy apparatus as claimed In any one of the 
preceding claims, wherein the detector means is a CCD camera. 

15 11. Confocal microscopy apparatus as claimed in any one of the 
preceding claims, wherein the mask is pattemed with equidistant light 
transmissive stripes and light barring stripes. 

12. Confocal microscopy apparatus as claimed in any one of the 
20 preceding claims, wherein the mask is has a random patterning. 

13. Confocal microscopy apparatus as claimed in any one of the 
preceding claims, wherein the mask is mounted for rotation about the 
optical axis of the apparatus. 

25 

14. A confocal microscopy method comprising providing Illumination 
from a first light source; encoding light incident on a specimen and 
decoding light from the specimen by means of one or more masks; 
generating a first image of the specimen consisting of a confocal image 

30 superimposed on a non-confocal image; and detecting said first image 
characterised by encoding light to or firom the specimen in reverse with 
respect to the encoding of the one or more mask and generating a second 
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image of the specimen consisting of a non-confocal image from which a 
confocai image has been removed. 

15. A confocai microscopy method as claimed in claim 14. further 

5 comprising generating a confocai image from said first and second images. 

16. A confocai microscopy method as claimed in claim 15. wherein the 
second image is subtracted from the first image to extract the confocai 
image. 

10 

17. A confocai microscopy method as claimed in any one of claims 14 to 
16, wherein a single mask is used to encode light to the specimen and to 
decode light from the specimen. 

15 18. A confocai, microscopy method as claimed in claim 17, wherein the 
second image is generated by reflecting light off reflective patterning on 
one surface of said single mask. 

19. A confocai microscopy method as claimed in any one of claims 14 to 
20 1 7, wherein the mask is rotated. 
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Figure 5a 




Figure 5b 
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Figure 5d 
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Figure 6b 
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Figure 6d 



